A few (ha!) words on assessment in higher ed

Like everyone else in U.S. higher education, of late, I have been compelled to talk and think a lot about “assessment.” This compulsion has picked up in intensity the past two years as Western went through an accreditation visit, and we are now dealing with the consequences of the report. On the whole, we came out well, but “assessment” in a number of areas was indicated as a weakness. Hence, the follow up thinking and talking.

For those outside the circle of academe, or those inside who are somehow insulated from the mania, the attention to “assessment” springs from increasing calls for “accountability” in higher education. We need to “prove” we're doing something useful, or at least that we do what we say we do in terms of educating students (and I would note that this is a movement that applies to the teaching end of what colleges and universities do, not the research and creative works side. I'm not entirely sure what to make of that yet, but there it is). “Assessment” in this context means specifying “outcomes,” that is, skills and practices we want students to acquire or become proficient in, and developing instruments for measuring whether those outcomes have been achieved.

So far so good and fair enough. However, like James Berger in Academe, I have concerns about the corporate manner in which this discussion is framed. At the same time, the underlying principles of this discussion are not only reasonable, but thinking about what we want our students to learn and why and how we can accomplish those purposes better is a good and right thing to do. And, of course, larger publics should be offered assurances or some kind of "proof" that we are doing right by students.

The problem is that faculty already do these things, and we do them all of the time (and, yes, I realize that there are faculty who probably don't, but that would be a small minority and what they do or don't do should not drive how colleges and universities are run). Being told that we need to “assess” student learning so we can be held “accountable” comes across as an insult because, at some level, it is.

Outside of the tensest moments, this insult is easy enough to put aside, not least because it frequently isn't intended. But that doesn't change the fact that the issue isn't really lack of assessment or even poor assessment, but the form that assessment takes and how the results can be and are communicated to others.

This isn't a trivial distinction. In discussing this issue at an administrative level, it is hard not to notice how often standardized tests come up as examples of good, or at least acceptable, assessment – ironic because faculty are exhorted to take this stuff seriously on the grounds that doing otherwise will mean federally mandated testing. Sounds like a Foucauldian nightmare/farce, right? It gets worse when the discussion turns to standardization of curriculum to make the testing possible.

What often seems lost in this discussion is a recognition that college and university faculty aren't professional teachers. We're professional scientists, writers, artists, etc. etc. who teach. This remains true even at institutions like mine where undergraduate education is the primary purpose. We're supposed to contribute to the development of knowledge, not just recite received works. Standardization runs counter to that larger process of knowledge production, stifles it even.

Fully standardized tests and curricula may be the extreme end of assessment, but the tendency seems built into a system based on interlocking layers of documentation declaring what it is we mean to do and how we'll let everyone know that we're doing it. This is at the heart of Jack Meacham's argument in favor of mission statements in higher education. Meacham sees these documents functioning like contracts, specifying the terms to which everyone (read: faculty, and maybe administrators) will be held to account.

Assuming that it's possible to come up with a statement that isn't so general as to be useless for anything meaningful, what do you do with work that falls on the margins or in between the gaps of the statement or that can't be easily boxed up in the first place? You could, of course, build something about free inquiry into your statement, but given that this is all meant to lead to “concrete” and “measurable” outcomes for students, it seems unlikely that such a principle would be acceptable in most contexts. Indeed, the whole exercise is designed to keep academics from ranging freely, or, put another way, the whole exercise is built on the assumption that faculty have too much freedom and need to reigned in.

(And, no, I don't think you can sequester academic freedom so that it only applies to research/scholarship, while teaching is encased in implicit contracts with accreditors, legislators, trustees, employers, “the community,” for specific “deliverables.” The two activities aren't supposed to be separate).

On a practical plane, process is often a stumbling block to making the enterprise work on its own terms. None of the conversations I've been party to have been particularly deliberative or deeply reflective. The geography department's current mission statement and learning outcomes were produced quickly in the hothouse environment of our institutional self-study. I suspect that this is the case for most departments at Western. That we were able to come up with reasonable documents in spite of the rush is suggestive of how essentially shallow these tasks are. They don't have to be, but the point here is that it is way too easy to just crank these things out, especially for people who spend a lot of their time working with words.

Don't misunderstand me. What we do, why we do it, what we want for our students, how we can best get what we want for our students, these are good, and necessary conversations, but, returning to James Berger, frameworks matter, reasons matter. Faculty already assess student learning for our own purposes. It's kind of inherent to the profession. No amount of “if we don't do it ourselves, someone else will do it for us” will change the fact that “someone else” is already framing the discussion (and interestingly, apparently, employers, often cited as the most important “someone elses” we need to be more responsive to, aren't, in fact, all that broken up about how colleges and universities are doing).

At Western, we're starting a review of our general education program. This is a two track issue. On one track is the corporate/accreditation imperative. By itself, that track could be basically empty, all paper, no practice. However, the other track is local. Are we doing what we want to do with gen ed? Could we be doing better? Ideally, the two tracks will meet at a satisfactory point. Similar to what Meacham claims on behalf of mission statements, the accreditors' report worked to open up a much needed conversation. And for that I'm thankful, but now the struggle begins to get and keep that conversation on the local track. This will be difficult because there will be no choice but to bring it back to the corporate station (sorry, had to finish the metaphor).

Let me reduce my argument to this: if someone can think of a way to draft a mission statement, outcomes, and assessment tools that can “measure” how close our students get to being able to produce and host This American Life in ways that would satisfy off campus constituencies, demanding as they are in our fevered imaginations, I'll renounce and cross-out this whole post.

Post-script:

After saving this the first time, I remembered an entry from Professor Zero where, echoing James Berger's piece cited above, she concisely sums up what is at stake in discussions about "assessment" and "accountability," and particularly in the way those issues are framed:

The view that the university should run and be run “like a business” is a right-wing model designed to produce obedient people and not critical thinkers. It is antithetical to the concept of the university as a place of intellectual community or, of course, as a self-governing community of scholars. I agree with these points.

In the same post, she makes note of a course she had to take as an undergraduate that only much later did she fully apprehend and appreciate. Is it possible to define and measure such an outcome? Maybe, but I suspect not in ways that can be tied to a corporate mission statement, or adequately quantified for a mid-term accrediting visit.

Comments

[Kerry]

I smiled at your nod to Ira Glass. He said something when I saw him live a few years ago that I have co-opted as a defense for liberal arts and general education. I forget the specific audience member's inquiry; it came from a student, and I think he was inquiring about Glass' pathway into the TAL mode of storytelling. Glass responded that "Ideas come from being around other ideas". Whenever I hear the "Why do I have to take.." refrain from students relating to our general education requirements, I talk about how learners and innovators look to new ideas to generate other new ideas.

[Anne-Marie]

Great post. I talked about something similar in terms of using someone else's standards to measure our impact a few weeks ago. That's something that really scares me.

As you know, I think there's a lot of good in what some of the learning-assessment people are saying - particularly those who are framing outcomes-based assessment as an alternative to the kind of standardized measurements prevalent in K-12. And those who are really struggling with the question of how we can demonstrate impact in those not-easily-measurable areas like critical thinking.

But at the same time, I really worry about how assessment efforts are actually implemented on a lot of campuses - as (frequently) unfunded mandates in which faculty and student affairs professionals are exhorted to "prove" that the university is delivering something that non-academics already think is important. That instead of really taking the time to define those crucial liberal arts, foundational, not-directly-related-to-employability things that we know to be important and focusing our assessment work on those goals.

In other words, if we (and the "we" here frequently refers to administration, not faculty -since administrators are the ones who usually have to defend what we do in higher ed to external stakeholders) aren't willing to do that hard thing, to make the case to those who don't already believe it that college graduates who can read, write, analyze, think, and understand cultural references on a broad, liberally-educated level are good for the world then assessment efforts become an exercise in jumping through someone else's hoops. And thats pretty scary.